System and method of lipstick bulktone and application evaluation

ABSTRACT

A method for predicting applied lipstick color includes extracting reference hue information from one or more digital images of a plurality of lip-tone cards, each lip-tone card including a plurality of lip-tone swatches, a white swatch, and a black swatch; extracting applied hue information from one or more digital images of a plurality of lip-tone cards coated with lipstick; and generating a predicted applied lipstick color based on the extracted applied hue information and one or more inputs indicative of a lipstick selection.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is based on and claims priority to U.S.Provisional Application No. 62/968,869, filed Jan. 31, 2020, thedisclosure of which is incorporated by reference herein in its entirety.The present application is also based on and claims priority to FrenchPatent Application No. FR 20 03394, filed Apr. 6, 2020, the disclosureof which is incorporated by reference herein in its entirety.

BACKGROUND

A user's lip tone can be one from a spectrum of possible lip tones,making it a challenge to predict the resulting hue of an appliedlipstick. Choosing the desired lipstick hue can be a difficult decisionfor a user, which is further compounded when the resulting hue of thelipstick is shifted after application to the user's lip. Thus, theuser's lip tone and the type of lipstick are contributing factors to anaccurate prediction. This can be additionally skewed based on otheraspects of the user's regimen, including applying other treatments tothe lips before and after the lipstick is applied, for example with alip balm, a lip gloss, or a lip liner. Furthermore, an applied pressureof the lipstick onto the user's lips can affect the resulting coverageand hue. Accordingly, better methods and systems for predictingresulting hue of applied lip cosmetics are desired.

The “background” description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent it is described in thisbackground section, as well as aspects of the description which may nototherwise qualify as prior art at the time of filing, are neitherexpressly or impliedly admitted as prior art against the presentinvention.

SUMMARY

The present disclosure relates to a method of predicting appliedlipstick color, including: extracting reference hue information from oneor more digital images of a plurality of lip-tone cards, each lip-tonecard including a plurality of lip-tone swatches, a white swatch, and ablack swatch; extracting applied hue information from one or moredigital images of a plurality of lip-tone cards coated with lipstick;and generating a predicted applied lipstick color based on the extractedapplied hue information and one or more inputs indicative of a lipstickselection.

The foregoing paragraphs have been provided by way of generalintroduction, and are not intended to limit the scope of the followingclaims. The described aspects, together with further advantages, will bebest understood by reference to the following detailed description takenin conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

A more complete appreciation of the disclosure and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1A shows a graph of a reflectance measurement, according to anexemplary embodiment of the present disclosure.

FIG. 1B shows a liptone card and a first application card, according toan exemplary embodiment of the present disclosure.

FIG. 2A shows additional lipstick applied to additional liptone cards,according to an embodiment of the present disclosure.

FIG. 2B shows a graph relating chroma and lightness, according to anembodiment of the present disclosure.

FIGS. 3-9 show an overview of the features of an embodiment fordetermining the liptone of a user.

DETAILED DESCRIPTION

FIG. 1A shows a graph of a reflectance measurement, according to anexemplary embodiment of the present disclosure. The graph relates areflectance of a material on a y-axis as a function of a wavelengthdetected on an x-axis. Any peak(s) exhibited by a trace in the graphdescribe hues predominantly reflected by the material. For a spectrumdetected in the visible range, for example between 350 nm and 800 nm,the peak correlates to the color of the material. For example, a peak at˜400 nm describes a blue material. For example, a peak at ˜700 nmdescribes a red material. Notably, constructive interference can be seenin the reflectance measurement if peak intensity increases for anadditional layer of material added (wherein the material transmitslight), as well as for a secondary peak exhibited at a multiple of thepeak wavelength. For example, a secondary peak for 400 nm blue lightemerges at 800 nm. The reflectance of the material is described asspecular or diffuse based on a smoothness of a surface of the material.Notably, these optical properties impact the appearance and design oflipsticks, and ultimately, their application by a user.

FIG. 1B shows a liptone card 105 and a first application card 105 a,according to an exemplary embodiment of the present disclosure. In anembodiment, the liptone card 105 includes a black swatch 110, a whiteswatch 115, and at least one liptone swatch 120. The at least oneliptone swatch 120 is an area on the liptone card 105 that represents aliptone of the user. These can vary and be selected based on liptonesmeasured from the users or predicted via emulation software andreproduced on the liptone card 105. For example, the at least oneliptone swatch 120 is printed on the liptone card 105. Similarly, theblack swatch 110 and the white swatch 115 are areas on the liptone card105 that represent a black and a white hue, respectively. The blackswatch 110 and white swatch 115 provide a reference point for which theat least one liptone swatch 120 is compared. The black swatch 110 has anRGB value that represents the hue black, for example (0, 0, 0), and thewhite swatch 115 has an RGB value that represents the hue white, forexample (255, 255, 255). An uncoated reflectance spectrum is determinedfor each of the at least one lipstone swatch 120. For example, anoptical spectrometer configured to scan through the visible light rangecan be used. Additionally, an optical image is obtained for each of theat least one liptone swatch 120. For example, a camera including anoptical sensor is used to capture the images.

In an embodiment, a first lipstick 125 is applied to the black swatch110, the white swatch 115, and each of the at least one liptone swatch120 to produce the first application card 105 a including a first coatedblack swatch 110 a, a first coated white swatch 115 a, and a first atleast one coated liptone swatch 120 a. For example, a hue of the firstlipstick 125 is substantially fuchsia. As shown in FIG. 1B, the liptonecard 105 includes six at least one liptone swatches 120 that progressfrom lighter near the white swatch 115 to darker near the black swatch110. Concomitantly, the six first at least one coated liptone swatches120 a coated with the first lipstick 125 a progress similarly fromlighter near the first coated white swatch 115 a to darker near thefirst coated black swatch 110 a. Since the first coated white swatch 115a and the first coated black swatch 110 a are also coated by the firstlipstick 125 a, the first coated white swatch 115 a can exhibit alightest tint for the first lipstick 125 a and the first coated blackswatch 110 a can exhibit a darkest shade for the first lipstick 125 a.The first at least one coated liptone swatches 120 a can exhibit varyingshifts in hue due to the contribution of the underlying liptone hue.

In an embodiment, a coated reflectance is determined for the firstcoated black swatch 110 a, the first coated white swatch 115 a, and eachof the first at least one coated liptone swatch 120 a. For example, theoptical spectrometer is used. Images are also obtained, for exampleusing the optical camera. The shift in the reflectance spectra from theuncoated to the coated reflectance is determined for each of theswatches. The change in the reflectance spectra from the white swatch115 to the first coated white swatch 115 a describes a shift in huewithout any additional contributions from the user's liptone, while alsodescribing a maximum lightness and a maximum chroma. Similarly, thechange in the reflectance spectra from the black swatch 110 to the firstcoated black swatch 110 a describes a shift in hue without anyadditional contributions from the user's liptone, while also describinga minimum lightness and a minimum chroma. Notably, the changes in thereflectance spectra from each of the at least one liptone swatches 120to each of the corresponding first at least one coated liptone swatches120 a can be compared to both the change in the reflectance spectra fromthe white swatch 115 to the first coated white swatch 115 a and from theblack swatch 110 to the first coated black swatch 110 a to determine theshift in hue due to contributions from the user's liptone. By the samemeasure, changes in lightness and chroma are also determined for each ofthe corresponding first at least one coated liptone swatches 120 a.Here, chroma describes a chromatic strength of an object's hue, or theperceived amount of difference from a grey of the same lightness, whilelightness describes a perceived brightness of the object's hue ascompared to a perfectly white object.

In an embodiment, a library of images of the first application card 105a including a wide range of the first at least one coated liptone swatch120 a is generated for the first lipstick 125 a and stored in a memorystorage. The hue, chroma, and lightness are also determined for each ofthe first at least one coated liptone swatch 120 a in the library andrespective spectra for each are stored in the memory storage.

FIG. 2A shows additional lipstick applied to additional liptone cards105, according to an embodiment of the present disclosure. In anembodiment, a second lipstick 125 b and a third lipstick 125 careapplied to the liptone card 105 to yield a respective second applicationcard 105 b and third application card 105 c. A hue of the secondlipstick 125 b is, for example, substantially scarlet, and a hue of thethird lipstick 125 c is, for example, substantially orange. The secondlipstick 125 b is applied to the black swatch 110, the white swatch 115,and each of the at least one liptone swatch 120 to produce the secondapplication card 105 b including a second coated black swatch 110 b, asecond coated white swatch 115 b, and a second at least one coatedliptone swatch 120 b. The third lipstick 125 b is applied to the blackswatch 110, the white swatch 115, and each of the at least one liptoneswatch 120 to produce the third application card 105 c including a thirdcoated black swatch 110 c, a third coated white swatch 115 c, and athird at least one coated liptone swatch 120 c. Similarly, the secondapplication card 105 b including a wide range of the second at least onecoated liptone swatch 120 b is generated for the second lipstick 125 band the third application card 105 c including a wide range of the thirdat least one coated liptone swatch 120 c is generated for the thirdlipstick 125 c, and all are added to the library. The correspondingreflectance spectra, chroma, and lightness are also determined andstored in the memory storage.

FIG. 2B shows a graph relating chroma and lightness, according to anembodiment of the present disclosure. In an embodiment, a trace isgenerated describing the chroma (y-axis) as a function of the lightness(x-axis) for each of the swatches for the first lipstick 125 a, secondlipstick 125 b, and third lipstick 125 c. While only six liptones areplotted in the graph of FIG. 2B for each lipstick, it may be appreciatedthat as more of the at least one liptone swatch 120 are coated andmeasured for each lipstick, the data in each of the traces becomes moregranular.

Advantageously, the library can be utilized to predict the hue of alipstick applied on the user's lips. In an embodiment, the lipstick isthe first lipstick 125 a and the user's liptone matches one of the atleast one liptone swatches 120. For example, the hue of the user'sliptone is within a predetermined deviation threshold of the hue of oneof the at least one liptone swatches 120. Thus, the correspondingreflectance measurement of the first coated at least one liptone swatch120 a for the first lipstick 125 a describes the hue of the appliedfirst lipstick 125 a on the at least one liptone swatch 120 that mostclosely matches the user's liptone.

In an embodiment, the user's liptone does not match one of the at leastone liptone swatches 120 in the library. That is, the hue of the user'sliptone is outside a predetermined deviation threshold of the hue of oneof the at least one liptone swatches 120. Here, interpolation betweentwo of the at least one liptone swatches 120 most closely matching theuser's liptone is performed to yield an interpolated liptone swatch thatmore closely matches the user's liptone. An interpolation between thetwo corresponding reflectance measurements for the two of the at leastone liptone swatches 120 (most closely matching the user's liptone) isalso performed to yield an interpolated hue for the applied firstlipstick 125 a. It may be appreciated that other targeted data in thelibrary can be interpolated, for example chroma and lightness. Due tothe diversity in users, collecting data on myriad liptones to expand thelibrary can improve the accuracy of predicting the hue, chroma, andlightness of a lipstick after application to the user's lips.

In an embodiment, the library can be sorted by a single liptone to matchthe liptone of the user. Thus, all of the lipsticks that have beenapplied to the at least one liptone swatch 120 that matches the user'sliptone are sorted. Subsequently, the user selects a desired appearance(hue) for the applied lipstick from the library and a determination ofthe lipstick to apply can be performed.

In an embodiment, the library does not include the matching liptone ofthe user. That is, the hue of the user's liptone is outside apredetermined deviation threshold of the hue of one of the at least oneliptone swatches 120. Again, interpolation between two of the at leastone liptone swatches 120 most closely matching the user's liptone isperformed to yield the interpolated liptone swatch. For example, theuser's liptone most closely matches a first at least one liptone swatch1201 and a second at least one liptone swatch 1201. Interpolationbetween the two corresponding reflectance measurements for all of thelipsticks applied to the first at least one liptone swatch 1201 and thesecond at least one liptone swatch 1201 is performed to predict theresulting hue of all the applied lipsticks on the interpolated liptoneswatch. For example, interpolation of the reflectance measurements for afirst of the first at least one coated liptone swatch 120 a 1 and asecond of the first at least one coated liptone swatch 120 a 2 isperformed to predict the hue of the first lipstick 125 a coated on theinterpolated liptone swatch. Similarly, interpolation of the reflectancemeasurements for a first of the second at least one coated liptoneswatch 120 b 1 and a second of the second at least one coated liptoneswatch 120 b 2 is performed to predict the hue of the second lipstick125 b coated on the interpolated liptone swatch. The same interpolationis performed on the third lipstick 125 c and all other lipsticks in thelibrary. All of the predicted coated hues of the lipsticks on theinterpolated liptone swatch can be filtered to yield predicted coatedhues within the predetermined deviation threshold of the user's desiredappearance (hue) for the applied lipstick, and a recommendation is madefor the lipsticks corresponding to the predicted coated hues within thepredetermined deviation threshold. While a lipstick yielding the exactuser's desired appearance may not exist, the user can be recommended aplurality of options that closely match the desired appearance (hue).For example, the user may select one of the recommended lipsticks and asecond recommendation can be made for a product to lighten the tint ordarken the shade of the recommended lipstick. For example, arecommendation is made for two or more lipsticks to mix coatings inorder to achieve the desired appearance (hue).

In an embodiment, the user selects a desired appearance (hue) for theapplied lipstick from the library but it does not match any entry in thelibrary. In this embodiment, the user's liptone matches one of the atleast one liptone swatches 120. An interpolation is performed betweenall the lipsticks coated on the matching at least one liptone swatches120 in order to determine at least two lipsticks that match the desiredappearance (hue) when coated on the matching user's liptone. Here, afirst recommendation is made for a lipstick that yields a slightlydarker shade and a second recommendation is made for a lipstick thatyields a slightly lighter tint when coated on the at least one liptoneswatch 120 that matches the user's liptone.

In an embodiment, the liptone card 105 includes additional swatches thatvary the hue, chroma, and lightness of the coated lipsticks. Forexample, the liptone card 105 includes additional at least one liptoneswatches 120 having lip balm applied thereon before coating with thelipsticks. For example, the liptone card 105 includes additional atleast one liptone swatches 120 for coating with the lipsticks followedby an additional coating of a second material, such as lip gloss, lipcream, a moisturizer, or any other top coat. For example, the liptonecard 105 includes additional at least one liptone swatches 120 forcoating with different types of lipsticks, for example a matte lipstick,a satin lipstick, a lip tint, a metallic lipstick, etc. For example, theliptone card 105 includes additional at least one liptone swatches 120for coating with a lipstick at varied application pressures, such aslight, normal, and firm pressures. Thus, for any one at least oneliptone swatch 120, myriad appearances for myriad lipsticks of the samehue can be catalogued and used to improve a user's lipstick selectionaccuracy and experience.

In an embodiment, the liptone card 105 includes artificial lips, forexample instead of the at least one liptone swatch 120 being printed onthe liptone card 105. Each of the artificial lips is pigmented accordingto diverse user liptones, and advantageously, are textured to resemblethe texture of real human lips, which affect the resulting appearance ofapplied lipsticks. For example, the artificial lips are molded collagengrown from skin keratinocytes grown on a scaffold, such as an acellular,nonimmunogenic dermal equivalent.

Subsequently, the aforementioned myriad coatings are coated onto theartificial lips. For example, lip balm is applied first, followed by thesatin lipstick, followed by the lip gloss. A reflectance measurement andoptical image can be obtained for each of the steps as each coating isapplied and stored in the memory as part of the library. Notably, thereflectance measured from the artificial lips provides a more diffusereflectance and realistic application environment as compared to a flatswatch.

FIGS. 3-8 illustrate additional details of features for (1) providingthe guidance to lipstick formulation on coverage (2) evaluating thecurrent lipstick application protocol and (3) creating models to predictlipstick application on different lip tones from the lipstick bulk tone.

The description set forth above in connection with the appended drawingsis intended as a description of various embodiments of the disclosedsubject matter and is not necessarily intended to represent the onlyembodiment(s). In certain instances, the description includes specificdetails for the purpose of providing an understanding of the disclosedsubject matter. However, it will be apparent to those skilled in the artthat embodiments may be practiced without these specific details. Insome instances, well-known structures and components may be shown inblock diagram form in order to avoid obscuring the concepts of thedisclosed subject matter.

Reference throughout the specification to “one embodiment” or “anembodiment” means that a particular feature, structure, characteristic,operation, or function described in connection with an embodiment isincluded in at least one embodiment of the disclosed subject matter.Thus, any appearance of the phrases “in one embodiment” or “in anembodiment” in the specification is not necessarily referring to thesame embodiment. Further, the particular features, structures,characteristics, operations, or functions may be combined in anysuitable manner in one or more embodiments. Further, it is intended thatembodiments of the disclosed subject matter can and do covermodifications and variations of the described embodiments.

It must be noted that, as used in the specification and the appendedclaims, the singular forms “a,” “an,” and “the” include plural referentsunless the context clearly dictates otherwise. That is, unless clearlyspecified otherwise, as used herein the words “a” and “an” and the likecarry the meaning of “one or more.” Additionally, it is to be understoodthat terms such as “upper,” “lower,” “front,” “rear,” “side,”“interior,” “exterior,” and the like that may be used herein, merelydescribe points of reference and do not necessarily limit aspects of thedisclosed subject matter to any particular orientation or configuration.Furthermore, terms such as “first,” “second,” “third,” etc., merelyidentify one of a number of portions, components, points of reference,operations and/or functions as described herein, and likewise do notnecessarily limit aspects of the disclosed subject matter to anyparticular configuration or orientation.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of this disclosure. For example, preferableresults may be achieved if the steps of the disclosed techniques wereperformed in a different sequence, if components in the disclosedsystems were combined in a different manner, or if the components werereplaced or supplemented by other components.

The foregoing discussion describes merely exemplary embodiments of thepresent disclosure. As will be understood by those skilled in the art,the present disclosure may be embodied in other specific forms withoutdeparting from the spirit or essential characteristics thereof.Accordingly, the disclosure is intended to be illustrative, but notlimiting of the scope of the disclosure, as well as the claims. Thedisclosure, including any readily discernible variants of the teachingsherein, defines in part, the scope of the foregoing claim terminologysuch that no inventive subject matter is dedicated to the public.

Embodiments of the present disclosure may also be as set forth in thefollowing parentheticals.

(1) A method of predicting applied lipstick color, comprising:extracting reference hue information from one or more digital images ofa plurality of lip-tone cards, each lip-tone card including a pluralityof lip-tone swatches, a white swatch, and a black swatch; extractingapplied hue information from one or more digital images of a pluralityof lip-tone cards coated with lipstick; and generating a predictedapplied lipstick color based on the extracted applied hue informationand one or more inputs indicative of a lipstick selection.

(2) The method of (1), wherein generating the predicted applied lipstickcolor includes generating a predicted applied lipstick color based onthe extracted applied hue information, the one or more inputs indicativeof a lipstick selection, and user lip hue information.

(3) The method of either (1) or (2), further comprising: extracting userlip hue information from one or more digital images; and generating thepredicted applied lipstick color based on the extracted applied hueinformation, the one or more inputs indicative of a lipstick selection,and the extracted user lip hue information.

(4) The method of (3), further comprising: generating one or moreinstances on a virtual display of a lipstick product predicted to resultin a hue, on the user, that is within a predetermined threshold of thehue of the lipstick selection, based on the extracted applied hueinformation, the one or more inputs indicative of a lipstick selection,and the extracted user lip hue information.

(5) The method of (3), further comprising: generating one or moreinstances on a virtual display of a predicted resulting lipstick hueassociated with the one or more inputs indicative of a lipstickselection, on the user, based on the extracted applied hue information,the one or more inputs indicative of a lipstick selection, and theextracted user lip hue information.

(6) The method of any one of (1) to (5), further comprising: determininga hue of the one or more inputs indicative of a lipstick selection.

(7) The method of any one of (1) to (6), further comprising: determininga lipstick having a hue that is within a predetermined threshold of thehue of the one or more inputs indicative of a lipstick selection.

(8) The method of (3), further comprising: determining a first liptoneswatch of the plurality of lip-tone swatch having the hue that is withina predetermined threshold of the hue of the lips of the user.

(9) The method of (7), further comprising: determining, based theapplied lipstick color corresponding to the lipstick, a recommendedlipstick.

(10) The method of any one of (1) to (9), further comprising:transmitting one or more of predicted applied lipstick color informationof a product based on the lipstick selection; and lipstick productrecommendation information regarding a product predicted to result in ahue, on the user, that is within a predetermined threshold of producthaving a hue of the lipstick selection.

(11) A method of predicting applied lipstick color, comprising:receiving a desired lip appearance from a user, the desired lipappearance having a target hue; determining a hue of lips of the user;determining, based on application of a lipstick applied to a pluralityof liptone swatches on a liptone card, a final hue of the appliedlipstick color, the plurality of liptone swatches having a hue within apredetermined deviation range of the hue of the user's lips; determiningthe lipstick correlating to the final hue that most closely matches thetarget hue of the desired lip appearance; and transmitting, to the user,information of the lipstick.

(12) The method of (11), wherein the step of determining the pluralityof liptone swatches having a hue within a predetermined deviation rangeof the hue of the user's lips further comprises: receiving the pluralityof liptone cards, each including the plurality of liptone swatches, awhite swatch, and a black swatch; determining a hue of each of theplurality of liptone swatches, the white swatch, and the black swatch;and determining a subset of the plurality of liptone swatches having thehue within a predetermined threshold of the hue of the user's lips.

(13) The method of (12), wherein the step of determining the lipstickcorrelating to the final hue that most closely matches the target hue ofthe desired lip appearance further comprises: coating the subset of theplurality of liptone swatches, the white swatch, and the black swatchwith the lipstick; determining the final hue of each of the coatedsubset of the plurality of liptone swatches; and determining thelipstick coated on the subset of the plurality of liptone swatcheshaving the final hue that most closely matches the target hue of thedesired lip appearance.

(14) The method of (13), further comprising: upon determining theliptone card does not include a matching liptone swatch having a huewithin the predetermined range of the hue of the user's lips,determining two matching liptone swatches in the plurality of liptoneswatches having hues closest to the predetermined range of the hue ofthe user's lips; determining the final hues of a plurality of lipsticksapplied to a first of the two matching liptone swatches and a second ofthe two matching liptone swatches; interpolating the final hues of theplurality of lipsticks applied to an interpolated liptone swatch havinga hue between that of the first of the two matching liptone swatches andthe second of the two matching liptone swatches; determining thelipstick correlating to the interpolated final hue that most closelymatches the target hue of the desired lip appearance.

(15) The method of (14), further comprising obtaining a simulated imageof the interpolated liptone swatch and a simulated image of theinterpolated liptone swatch coated with the plurality of lipsticks.

(16) The method of (13), further comprising normalizing the determinedfinal hues of each of the coated subset of the plurality of liptoneswatches against the determined final hues of the black swatch coatedwith the lipstick and the white swatch coated with the lipstick.

(17) The method of (13), further comprising obtaining an image of thelips of the user and an image of the coated subset of the plurality ofliptone swatches, wherein the hue of the user's lips is determined fromthe image of the user's lips and the final hue of each of the coatedsubset of the plurality of liptone swatches is determined from the imageof the coated subset of the plurality of liptone swatches.

(18) The method of (17), further comprising storing, in a memory, thehues, the final hues, and the images of each of the uncoated and coatedsubset of the plurality of liptone swatches, the white swatch, and theblack swatch.

(19) The method of (18), wherein determining the applied lipstick coloris determined based on a trained neural network.

(20) The method of (19), wherein the neural network is trained using atraining dataset, the input data of the training dataset including thehues, the final hues, and the images of each of the uncoated and coatedsubset of the plurality of liptone swatches, the white swatch, and theblack swatch.

(21) A method of predicting applied lipstick color, comprising:receiving a lipstick selection from a user, the lipstick having a hue;determining a hue of lips of the user; determining, via a liptone cardhaving the lipstick selection applied thereon, a final hue of the user'slips having the lipstick selection applied; and transmitting to the userthe final hue of the user's lips.

(22) The method of (21), wherein the step of determining the final hueof the user's lips having the lipstick selection applied furthercomprises: receiving the plurality of liptone cards, each including theplurality of liptone swatches, a white swatch, and a black swatch;determining a hue of each of the plurality of liptone swatches, thewhite swatch, and the black swatch; and determining a matching liptoneswatch having a matching hue within a predetermined threshold of the hueof the user's lips; and applying the lipstick selection to the matchingliptone swatch.

(23) The method of (22), wherein upon determining the liptone card doesnot include a matching liptone swatch having a hue within thepredetermined range of the hue of the user's lips, determining twomatching liptone swatches in the plurality of liptone swatches havinghues closest to the predetermined range of the hue of the user's lips;and interpolating the final hue of the lipstick selection applied to aninterpolated liptone swatch having a hue between that of the first ofthe two matching liptone swatches and the second of the two matchingliptone swatches.

(24) The method of (23), further comprising obtaining a simulated imageof the interpolated liptone swatch and a simulated image of theinterpolated liptone swatch coated with the lipstick selection.

(25) The method of any one of (22) to (24), further comprisingnormalizing the determined final hues of each of the coated plurality ofliptone swatches against the determined final hues of the black swatchcoated with the lipstick selection and the white swatch coated with thelipstick selection.

(26) The method of any one of (22) to (25), further comprising obtainingan image of the lips of the user and an image of the coated plurality ofliptone swatches, wherein the hue of the user's lips is determined fromthe image of the user's lips and the final hue of each of the coatedplurality of liptone swatches is determined from the image of the coatedplurality of liptone swatches.

(27) The method of (26), further comprising storing, in a memory, thehues, the final hues, and the images of each of the uncoated and coatedplurality of liptone swatches, the white swatch, and the black swatch.

(28) The method of (27), wherein determining the applied lipstick coloris determined based on a trained neural network.

(29) The method of (28), wherein the neural network is trained using atraining dataset, the input data of the training dataset including thehues, the final hues, and the images of each of the uncoated and coatedplurality of liptone swatches, the white swatch, and the black swatch.

1. A method of predicting applied lipstick color, comprising: extractingreference hue information from one or more digital images of a pluralityof lip-tone cards, each lip-tone card including a plurality of lip-toneswatches, a white swatch, and a black swatch; extracting applied hueinformation from one or more digital images of a plurality of lip-tonecards coated with lipstick; and generating a predicted applied lipstickcolor based on the extracted applied hue information and one or moreinputs indicative of a lipstick selection.
 2. The method of predictingapplied lipstick color of claim 1, wherein generating the predictedapplied lipstick color includes generating a predicted applied lipstickcolor based on the extracted applied hue information, the one or moreinputs indicative of a lipstick selection, and user lip hue information.3. The method of predicting applied lipstick color of claim 1, furthercomprising: extracting user lip hue information from one or more digitalimages; and generating the predicted applied lipstick color based on theextracted applied hue information, the one or more inputs indicative ofa lipstick selection, and the extracted user lip hue information.
 4. Themethod of predicting applied lipstick color of claim 3, furthercomprising: generating one or more instances on a virtual display of alipstick product predicted to result in a hue, on the user, that iswithin a predetermined threshold of the hue of the lipstick selection,based on the extracted applied hue information, the one or more inputsindicative of a lipstick selection, and the extracted user lip hueinformation.
 5. The method of predicting applied lipstick color of claim3, further comprising: generating one or more instances on a virtualdisplay of a predicted resulting lipstick hue associated with the one ormore inputs indicative of a lipstick selection, on the user, based onthe extracted applied hue information, the one or more inputs indicativeof a lipstick selection, and the extracted user lip hue information. 6.The method of predicting applied lipstick color of claim 1, furthercomprising: determining a hue of the one or more inputs indicative of alipstick selection.
 7. The method of predicting applied lipstick colorof claim 1, further comprising: determining a lipstick having a hue thatis within a predetermined threshold of the hue of the one or more inputsindicative of a lipstick selection.
 8. The method of predicting appliedlipstick color of claim 3, further comprising: determining a firstliptone swatch of the plurality of lip-tone swatch having the hue thatis within a predetermined threshold of the hue of the lips of the user.9. The method of predicting applied lipstick color of claim 7, furthercomprising: determining, based the applied lipstick color correspondingto the lipstick, a recommended lipstick.
 10. The method of predictingapplied lipstick color of claim 1, further comprising: transmitting oneor more of predicted applied lipstick color information of a productbased on the lipstick selection; and lipstick product recommendationinformation regarding a product predicted to result in a hue, on theuser, that is within a predetermined threshold of product having a hueof the lipstick selection.